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ABSTRACT 



This study compares the effectiveness of on-the-job and 
formal training methodologies of United States Coast Guard 
enlisted rate training as a function of job performance and 
rate of advancement. Results indicate there is no 
significant difference in evaluated work performance between 
"A" school and striker graduates at the E4 and E5 rank. The 
comparison of rate of advancement to E5 indicates the 
Boatswains Mate (BM> and Machinery Technician <MK> personnel 
undergoing the striker, on-the-job training advance 
significantly faster than their "A" school counterparts. Th 
difference in rate of advancement between Yeoman (YN) and 
Storekeeper (SK) strikers and "A" school graduates was not 
found to be statistically significant. Recommendations are 
made to perform a cost effectiveness study, review selection 
criteria for "A" schools and establish striker program 



standards . 
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INTRODUCTION 



I . 

As the Coast Guard enters the 1990's, the organization Is 
faced with uncertainty regarding level of funding as a result 
of the federal deficit and political processes. This reality 
forces an evaluation of the strategy, goals and plans of 
operation required to meet our missions. One Important 
aspect of the organization is the development of our human 
resources, which makes them capable of performing jobs that 
collectively define Coast Guard missions. As a personnel 
intensive service, it is a mandate that our members are 
trained in the most efficient and effective manner possible. 
There is no single "right" way to train, but there are always 
more effective and efficient methods and programs by which we 
can establish the knowledge, skills and abilities of Coast 
Guard personnel to perform their jobs to standards. 

Personnel productivity and job performance are difficult 
to quantify in a service organization. If these indicators 
are analyzed in the context of the organization's strategy 
and scope of the study made, they can provide Information to 
aid decision making with respect to resource allocation. 

This paper will address the effectiveness of the two 
initial enlisted rate training methodologies; (1) formal, 
resident "A" school, and (2) informal, non-resident, 
on-the-job training (the striker program). A comparison will 
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be made of job performance measures of personnel trained by 
the two methods. 

There are no prior assumptions about which training 
method should result in superior performance because there 
are many variables that can impact on performance and which 
are difficult to control. The question of which method is 
more effective becomes interesting in light of research that 
shows that programmed instruction is much more efficient than 
on-the-job training in respect to knowledge acquistion. The 
next question then becomes: how effectively are personnel 
performing their missions on-the-job and does the 
effectiveness relate to training methodology? 

The external evaluation of existing training programs is 
important if we are to assess the real Impact of our 
programs. Opinions of Coast Guard supervisors on the subject 
of training effectiveness are as diverse as the unit 
implementation plans of the striker program. There is 
certainly a place for both formal, classroom training and 
informal, on-the-job training as vehicles to shape behavior 
and develop knowledge, skills and abilities in our personnel. 
This study may assist in determining the proper mix of 
training resources we apply to our human capital to achieve 
the goals we set for the organization as a whole. 
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BACKGROUND 



II . 

This chapter will outline characteristics and procedures 
of the two general Coast Guard <CG> enlisted rate training 
programs and provide an overview of the assignment and 
advancement processes. 



A. GOALS AND CHARACTERISTICS OF COAST GUARD ENLISTED RATE 

TRAINING 

The goal of Coast Guard rate training is to "...provide 
military personnel with the capability to better fulfill 
Coast Guard missions and their necessary support." <CG 
Personnel Manual (PERSMAN), pp . 1-2.) The specific 
objectives are: 

o To indoctrinate personnel with an understanding of their 
assigned duties, to develop necessary skills and to 
develop a sense of responsibility within the service 
organ i zat i on . 

o To develop personnel through a balanced program of 

formal, resident and on-the-job training to reasonably 
ensure the safe and competent execution of their 
assigned tasks. 

o To provide personnel with an opportunity to qualify for 
promotion and advancement. 

There are currently two methods of attaining the specific 
CG enlisted ratings described in Appendix A. An enlisted 
person can attend a formal, resident "A" school or engage in 
an informal , non-resident striker program. The 
characteristics of each are described below: 
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1. "A" 



School 



"A" School is a formal, classroom-oriented program of 
instruction which is specific for each enlisted rate. "A" 
schools are located in Petaluma, CA; Yorktown, VA; and New 
London, CT and are characterized by: 

o Standardized Instruction. Lesson plans corresponding to 
terminal performance skill or knowledge objectives are 
developed for each course and presented in a relatively 
consistent manner to each class. 



o 



"Trained" 
on their 
training 



Instructors. Instructors are selected based 
job performance and attend an instructor 
class to learn effective teaching techniques. 



o Controlled Environment. "A" school is conducted in a 
well defined and controlled classroom, facility or shop 
setting which attempts to minimize external interference 
and control the training process. 

o Resident Program. "A" school attendees are assigned 
specifically to the "A" school for instruction and 
usually reside in a student barracks. 



o 10-16 Week Duration. Courses are composed of lesson 
plans which cover the practical factors required for 
advancement to E4 in a specific time frame. 



a. "A" School Selection 



Selection to "A" school is based upon: 



o 



Minimum Armed Services Vocational Aptitude Battery 
(ASVAB) composite scores as shown in Table 1. 



o Informal assessment of individual motivation by 
Commanding Officer to complete and make use of the 
tra i n i ng . 



o No non-judicial punishment or civil convictions for six 
months prior to applying for "A" school. 

o Special requirements for sight, hearing, security 
clearance or citizenship. 
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"A" SCHOOL REQUIREMENTS 



TABLE 1 



School Current 



Min lmum 



Rate 


Duration 


Wait Time 


ASVAB Compos i tel 


BM 


102 


253 


VE+AR=1 0 1 


DC 


13 


18 


VE=MC+AS=1 52 


EM 


16 


16 


MK+EI +GS=152 & AR=52 


MK 


14 


9 


AR=MC=AS=1 50 or VE+AR=106 


OM 


12 


2 


VE+AR=1 10 & N0+CS=1 0 1 & AR=57 


SK 


10 


31 


VE+AR=10 1 


SS 


13 


2 


VE+AR=1 0 1 


YN 


1 1 


9 


VE+AR=106 & N0+CS=1 0 1 



*CG Training and Education Manual, pp . 216-219. 
^School Duration in weeks (as of Jun 89). 

3 Current Wait Time in months for last E3 on list. 
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The actual number of "A" school graduates per 
year, per rate is determined by Commandant, Office of 
Personnel <G-P), and is based on the number of guaranteed 
school requirements, expected separations from the service, 
expected number of strikers who make rate and the needs of 
the service. The waiting list for "A" school is publicized 
Coast Guard wide and is also shown in Table 1. Coast Guard 
policy states that within six months of reporting to one's 
first unit out of boot camp, the member should submit an "A" 
school request which establishes his position on the "A" 
school waiting list for that rate. He will continue to 
perform his assigned duties as an E2 or E3 until he is picked 
up for school . 

b. "A" School Graduates 

"A" school graduates are expected to have basic, 
essential rate knowledge and be able to perform technical 
skills at the E4 level of performance as outlined in the 
Enlisted Qualification Codes Manual <EQCM). An excerpt from 
the EQCM for the Electricians Mate (EM) rate is included as 
Appendix B. Class "A" school graduates are apprentices who 
will require on-the-job training under close supervision to 
carry out duties. Upon successful completion of the "A" 
school course of instruction, the member is normally advanced 
to E4 if all requirements for advancement are met (l.e., 
successful completion of military requirements, 
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correspondence course and length of active duty 
requ i rements) . 

2. The Striker Program 

The striker program is an informal program of 
instruction conducted at the member's parent unit. It is 
available for the rates which have been assessed by the 
headquarters force manager as being relatively non-techn i ca 1 
and "teachable" on the job. Those rates are BM, DC, EM, MK, 
QM , SK, SS and YN. The instruction methodology is 
characterized by: 



o Self-Paced, Individualized Instruction. Striker 

programs are self-paced and dependent upon the resources 
available at each unit. 

o Non-Dedi cated Instructors. Rated personnel assigned to 
each CG unit can act as on-the-job instructors to the 
striker, but they have other duties to perform and do 
not normally have access to an instructor training 
course . 

o Normal Working Conditions. The striker program must be 
integrated with the operational commitments of the unit. 
This may allow for a full, five day striker program or 
it may only allow two, half-day training periods per 
week . 

o Unspecified Course Duration. There is no specific time 
required by the CG to complete a striker program. 

o Non-Standard Instruction or Curriculum. There are no 
terminal learning objectives associated with the striker 
program. As such, it is an ad hoc curriculum which 
usually results from the needs of the striker and 
department head of the rate being struck. 

Strikers must show proficiency of each required 
practical and knowledge factor in their rate for the E4 level 
of expertise as specified in the EQCM . Each factor is 
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verified and annotated by a subject matter expert at the 
unit, and later reviewed by the Commanding Officer. 

Striker programs vary in their implementation, but 
are all based on the completion of the practical and 
knowledge factors required by the EQCM. The amount of time 
taken to complete the practical factors and compete for 
advancement in the striker program are dependent upon: 
o Unit operational requirements, 
o Command and departmental priorities, 
o Individual motivation and ability. 

B. ASSIGNMENT PROCESS 

Following the completion of "A" school or the striker 
program, the member is advanced to E4. From that point, the 
needs of the service dictate his location of service in the 
Coast Guard. Graduates of both programs are expected to 
perform at the E4 level of expertise in their rating and are 
eligible for assignment to any Coast Guard unit. While 
strikers may stay at the unit where they have performed their 
on-the-job training, there is no guarantee of this. Rate 
vacancies are the driving force behind the assignment 
process. Conversely, “A" school graduates would return to 
their parent units upon E4 designation only if a vacancy 
exists. It is assumed that there is no bias towards either 
group in the transfer process. In other words, the detailers 
are concerned about previous unit experience in relation to 
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qualifications and accrued "sea" time, but the variable of 
rate training method is not considered in the assignment 
process . 

C. ADVANCEMENT PROCESS 

The enlisted advancement process is detailed in Chapter 5 
of the Coast Guard Personnel Manual. The objectives of the 
advancement process are: 

o To allow an orderly progression of enlisted personnel in 
the rating structure. 

o Ensure the required degree of proficiency at the various 
grade levels within each rate. 

o Promote those best qualified to fill vacancies which 
occur . 

Advancement is based on a system of servicewide 
competition. To qualify to compete in the servicewide 
examination (SWE), which is given for all rates (E4 through 
E6) twice a year in March and September, each member must: 

o Fulfill length of active duty requirements. 

o Fulfill minimum sea duty requirements. 

o Successfully complete military requirements and rate 
technical correspondence courses. 

o Complete all practical and knowledge factors for the 
rating considered. 

o Obtain a parent command recommendation for advancement 
which is based on the member's assessed qualities of 
character and leadership potential. 

o Have at least minimum enlisted evaluation marks for the 
period prior to the servicewide exam. 
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The advancement criteria is a composite servicewide 
examination score which is detailed in Table 2. The 
composite score is representative of each member's 
performance, seniority and knowledge. (PERSMAN, 5-C-2) 

1 . Job Performance 

The performance factor is a cumulative measure of the 
member's performance at his current rate. The score is 
derived from the Enlisted Performance Evaluation form 
(Appendix C>. This evaluation is conducted twice a year by 
each member's supervisor and the performance criteria are 
measured on a behavioral ly anchored rating scale. It is an 
assessment of the member's total output or productivity to 
the Coast Guard as measured by military, team work, 
leadership, service representative and human quality factors. 

2. Seniority 

Seniority is quantified by assessing time in service 
and time in paygrade. The more job experience, the higher 
the weight this factor holds in the advancement process. 

3. Knowledge 

The knowledge criteria for the servicewide 
competition is assessed by the servicewide examination given 
twice a year in March and September. The exam is content 
valid with respect to the knowledge and practical 
qualifications required for the next higher rate. The 
subject matter content is divided into pass/fail and rank 
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SERVICEWIDE COMPETITION SCORE 



TABLE 2 



Maximum Score l 



Exam Score 80 

Performance Factor 50 

Time in Service2 20 

Time in Paygrade3 20 

Medals & Awards^ i q 

Maximum Composite 180 

SWE Score 



% of Total Composite 
44 
28 
1 1 
1 1 

6 

100 



1 CG PERSMAN, p. 5-C-2 

^Time in Service (TIS), one point per year 

•^Tirne in Paygrade (TIR), two points per year 

^Medals & Awards, points determined by award earned as 
specified in CG PERSMAN, p. 5-C-2. 



ordering questions. To be placed on an advancement 
eligibility list, the candidate must achieve a passing score 
on the pass/fail portion of the test. The rank ordering 
questions are intended to distribute the minimum and highly 
qualified candidates. 

4. Precedence List for Advancement 

The total servicewide examination composite is used 
to establish a precedence list for advancement of eligible 
personnel. A cutoff is established for every list, for every 
rate and is based on the anticipated number of rate vacancies 
in the next six month period. If above the cutoff, the 
member is assured advancement. If below the cutoff, the 
member will not likely be picked up for advancement and must 
again compete in the next servicewide examination. This 
advancement process is used for advancement to E4, E5 and E6. 
The only exception is at the E4 level where "A" school 
graduates do not compete in the servicewide examination, 
although they do successfully complete terminal performance 
objectives for the course of instruction. Strikers who pass 
the servicewide examination for E4 are generally advanced to 
E4 when vacancies are available as the entire eligibility 
list is considered above the cutoff. The first time strikers 
and "A" school graduates compete against each other for 
advancement is at the E5 level. 
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THEORETICAL FRAMEWORK 



The goal of any training program is to change or 
establish a behavior which is required for the accomplishment 
of specific tasks of a Job a person will be performing. This 
is successfully accomplished when the trainee posesses the 
knowledge, skill or ability to perform the required tasks, 
on-the-job, to an established level of proficiency. 

This chapter will establish the theoretical significance 
of initial rate training and subsequent advance, and discuss 
the relation it has with job performance and productivity. 

A. SIGNIFICANCE OF INITIAL RATE TRAINING 

Initial rate training is a member's indoctrination to a 
specific work discipline which differs from other specialty 
rates within the organization. It has the following impacts 
on the i ndi v i dual : 

1. Establishes Conceptual Framework 

Each rate has a specific mission and a unique theory, 
body of knowledge and tasks which define it. For most 
trainees, their first involvement with the concepts of each 
rate is during their training as a striker or an "A" school 
attendee. Studies have shown that problem solving is 
facilitated by previously presented information which was 
presented in a form that required for its comprehension, the 
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same conceptual processing as that demanded by the subsequent 
problem. (Lockhart, Lamon , Gick, 1988, p.36> Additionally, 
the manner in which an individual is taught to perform tasks, 
or gain a body of knowledge, or acquire problem solving 
skills affects the ability to make judgments and perform in 
the future. Initial skill acquisition is important in the 
development of judgment relating to the use of a set of 
rules. (Carlson and Schneider, 1989, p.240) 

2. Scope of Job 

Initial rate training will focus the trainee's 
attention on the requirements of the rate and will establish 
role perception. Expectations for the trainee's behavior and 
attitude as a professional petty officer are conveyed to him 
through the training, and is largely a function of 
characteristics of the trainer, the environment of the 
training and the course of instruction. (Goldstein, 1986, 
pp. 88-106 and Kirkpatrick, 1985, pp. 302-319) 

3. Performance Standards 

Initial rate training sets the standards of 
performance expected of a rated petty officer. These 
standards will be used at a later date in the evaluation 
process, which attempts to assess all aspects of a member's 
performance and his or her contribution to the organization. 

4 . Validity 

In order to be successful, initial rate training must 
be content valid in relationship to the actual job 
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requirements. It must teach to a set of job performance 
standards required in the field. The more valid the training 
program, the greater the chance of effectively establishing a 
conceptual framework in an individual which will provide the 
basis for an acceptable level of job performance. 

B. SUBSEQUENT ADVANCED RATE TRAINING 

Following initial rate training, the member will acquire 
new knowledge and skills from the unit training program, his 
job experiences, advanced "C" school training and 
correspondence courses. This subsequent training will either 
reinforce current behavior, provide new knowledge and skills, 
or contradict the initial training. The further one travels 
chronologically from the initial training, the greater the 
potential for cumulative interference with the initial 
framework already established and the more difficult it 
becomes to establish cause and effect between initial 
training method and observed job performance. (Freda, Hall, 
Ford, 1981 , p.419> 

C. PRODUCTIVITY AND JOB PERFORMANCE 

The human captial interpretation of the experience 
earnings profile states that individuals are paid in direct 
relation to their productivity, or their earnings for the 
organization. As an employee receives more training, human 
capital is increased and more is provided to the organization 
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in the way of output. Since people increase their knowledge 
and skills through work experience and formal and informal 
instruction, the potential for increased productivity is 
rewarded by advancement to the next higher paygrade. Thus, 
it is assumed that the more quickly a person advances, the 
more productive he or she is for the organization. 
Productivity is a relative term in a service organization and 
is very difficult to measure in terms of a person's output. 
(Medoff, 1980, p.703) In the context of this study, a 
person's total productivity is given as a function of the 
quality and quantity of work performed, the contribution to 
the well being of the unit and the professional image he 
conveyed to the public. 

The Enlisted Performance Evaluation form attempts to 
capture various measures of productivity of the individual 
with six composite factors reflecting military manner, team 
effectiveness, work ability and accomplishments, leadership 
ability. Coast Guard representation and human factors. 

Within each division of a unit, supervisors mark an 
individual's performance in relation to descriptors of each 
aspect of a person's performance on a scale of one to seven. 
These rankings will reflect the overall productivity to the 
service in an ideal sense. The supervisor will be assessing 
the individual's problem solving capability, ability to work 
with others, stamina, conduct, motivation and workmanship, as 
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incorporated in this Enlisted Performance Evaluation form 
shown as Appendix C. 

Specifically, the work performance factor should be 
dependent to a large degree on the initial training the 
person received. It would be theorized that, all other 
conditions held equal, a person with a superior conceptual 
framework, knowledge and skills will have a higher work 
factor which is a reflection of higher productivity. A 
person with this background will also advance more quickly as 
a result of a higher servicewide evaluation composite based 
on superior performance. 
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IV. LITERATURE REVIEW 



Past studies and experiments have been performed which 
attempted to assess training effectiveness of various 
programs of instruction. The problems associated with them 
are common to the problems associated with any measurement of 
behavior in the social sciences; that is, how to account for 
threats to external and Internal validity. This section will 
provide an overview of previous studies as they pertain to 
the concepts of training effectiveness and efficiency. 

A. TRAINING EFFECTIVENESS 

Training is conducted with the intent of enabling a 
person to perform tasks and gain knowledge needed for the 
performance of a job at a later date. Kirkpatrick, (1985, p. 
302) assesses the effectiveness of a training program at four 
different levels: 

o Trainee react i on to training, 
o Trainee 1 earn i no during training. 

o Trainees change in behavior as a result of training, 
o Trainees product i v i tv as a result of training. 

While all four levels are important in an overall 
evaluation of training, behavior as assessed by actual job 
performance is a valid and significant indicator of training 
effectiveness. Vineberg, Joyner and Taylor (1978, p.2) 
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suggest that training effectiveness is a function of the 
number of tasks taught and the resultant quality of 
performance. Other studies have elaborated on the complexity 
of training effectiveness as a function of characteristics of 
the individual, the characteristics of the programs of 
instruction and conditions of the workplace. 

1. Characteristics of the Individual 

Characteristics of the individual which influence job 
performance are ability, motivation and role perception. 
(Porter and Lawler, 1968 cited by Terborg, 1977, pp. 188-216) 
a . Trainee Ability 

Porter and Lawler (1968, cited by Terborg, 1977, 
pp. 188-216) included the characteristics of the individual in 
their model explaining job performance. Subsequent field 
studies have shown that, where ability test scores were used 
in selection decisions, there was relatively little support 
for the hypothesized relationship between ability and job 
performance. (Gavin, 1970; Lawler and Suttle, 1973; Vroom, 
1960, p.189 cited by Terborg, 1977, p.189) This finding is 
supported by subsequent studies that show ability level of 
graduates. .. (of a training program) are... not related to 
fleet performance measures, but do have predictive power for 
school performance measures. (Freda, Hal, Ford, 1981, p.419) 
The ASVAB minimum scores for selction to enlisted 
"A" schools have been shown to be good predictors for 
performance in training, but are less valid as predictors of 
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actual job performance in the field. (Abellara, 1976, cited 
by Eitelberg, 1988, pp. 80-81) Achievement in the cognitive 
realm does not ensure achievement in the performance realm. 
(Sullivan and Elenburg, 1988, pp. 38-43) 

b. Trainee Motivation 

Motivation is the individual's desire to 
demonstrate a known behavior (or learn a new one) and 
reflects willingness to expend effort. (Muchinsky, 1987, 
p .446) There are numerous theories which attempt to model 
individual motivation. All have met with various degrees of 
empirical support, but no theory has received so much support 
that it can be declared the theory of motivation. (Muchinsky, 
1987, p.484) A balanced perspective would indicate that 
there are personal traits and environmental variables which 
impact on an Individual's desire to put forth effort in the 
accomplishment of a job, or even to learn how to perform the 
job in the first place. 

c. Trainee Role Perception 

Role perception is defined as the direction of 
effort on what action or behavior the individual thinks is 
appropriate for the task. (Porter and Lawler, 1968 cited by 
Terborg, 1977, p.188) This aspect of human behavior is 
unique to each individual and corresponds closely to the idea 
of merging individual judgment and motivational 
character i st i cs . 
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2. Characteristics of the Program of Instruction 

Characteristics of the program of instruction which 
influence job performance include: 
o Training methodology, 
o Training validity. 

o Instructor expectations. (Goldstein, 1986, pp. 88-106) 
a. Training Methodology 

Instructional methodology has long been thought 
to influence the effectiveness of a training program. Miner, 
Das and Gale (1984, pp. 49-59) assessed three different 
training methodologies (case oriented, experiential and 
lecture) aimed at teaching organizational behavior. Their 
findings indicate there was no significant difference between 
the methodologies as measured by a final exam, a post exam 
(one month after course completion) or student satisfaction 
with the course. They recognized a threat to the external 
validity of their model in the amount of effort spent 
reviewing each other's course material and test instruments, 
which could have attenuated some of the "natural" differences 
between the methods. 

A mail survey of training experts conducted by 
L.L. Neider (1981, pp. 24-30) ranked training methods by the 
criteria of satisfying a particular objective. The results 
of the survey indicate lecture and programmed instruction 
were regarded as the most effective methodologies with 
respect to knowledge acquisiton, with lecture ranking ahead 
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of programmed instruction in the areas of changing attitudes, 
problem solving skills, interpersonal skills and participant 
acceptance. Programmed instruction ranked ahead of lecture 
in knowledge retention. Whatever the training or teaching 
methodology, learner involvement may be a key variable in 
training effectiveness. Esbeck <1984, pp. 71-83) proposes 
that the learner's Involvement in the learning process 
affects his productivity in much the same way that a worker's 
increased participation and responsibility in production in 
the company results in a significant improvement in 
product i v i ty . 

Training methodology also includes the variable 
of time. The amount of time spent learning a task and time 
between practice sessions are significant determinants in 
skill acquistion and retention. Goldstein <1986, p.83) 
citing Underwood <1964) has shown that massed practice is 
better than spaced practice for acquisiton <of verbal 
skills), but poorer for retention. The literature indicates 
that distributed practice utilizing reasonable rest periods 
is the favored technique for optimization of performance. 
<Goldstein, 1986, pp. 80-87) 

b. Training Validity 

In order for training to be effective, trainees 
should be using the skills, knowledge and attitudes that are 
developed in training, in the transfer setting. <Goldstein, 
1986, p.88) With on-the-job training, the training and 
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transfer setting are Identical and positive transfer has been 
shown to occur when task stimuli and required responses are 
Identical In both settings. (Holding, 1965, p.90 cited by 
Goldstein, 1986) However, most training and transfer 
settings are not the same and the degree of congruence 
between the task stimuli and the required response in both 
settings is a measure of the training validity of the 
program. 

c. Instructor Expectations 

A significant factor in the success or failure of 
any training program is the instructor. Eden, Ravid and 
Shani (1982, cited by Goldstein, 1986, p.106) performed 
controlled experiments that indicated that inducing high 
expectations of students in trainers enhances trainee 
performance. This relates closely to the motivational 
qualities of the instructor. 

3. Conditions of the Workplace 

Conditions of the workplace include supervisor 
support, feedback and rewards. (Kirkpatrick, 1985, pp . 
302-319) The trainee will walk away from the course of 
instruction ready, willing and able to perform his Job at a 
given degree of proficiency. The conditions of the workplace 
will either enable or disable the trainee to carry out his 
duties by the support, feedback and rewards available. 

Studies dating back to the Hawthorne Effect in 1930 account 
for increases in productivity as a result of personal 
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attention paid to workers by their supervisors. (Dwyer, 1977, 

p . 8) 



B. TRAINING EFFICIENCY 

While this paper deals primarily with training 
effectiveness, the efficency of training in relation to 
learning achieved and program cost is important enough to be 
addressed in this chapter. 

Studies of training efficiency have found that when 
comparing on-the-job training (OJT) to structured training 
programs, OJT is not as efficient in teaching employees 
necessary skills. (Kainer, Begley, Maggard, 1983, pp. 84-95) 
They further show that OJT is the least efficient training 
methodology available in history. 

Marcus and Quester <1986, p.13) measured net productivity 
of a United States Navy trainee as the contribution of the 
trainee less the loss in productivity of the trainer in the 
training process. They found that when camparing "A" school 
and striker training with this research methodology, "A" 
school training was more cost efficient as a result of a 
greater net productivity associated with "A" school 
graduates. The productivity assessments were made on the 
basis of supervisor ratings which compared strikers and "A" 
school graduates to an average four year rate specialist in 
the Navy. 
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A previous Coast Guard study (Langholtz, 1979, pp . 66-67) 
found the opposite to be true In a cost analysis of resident 
versus correspondence training in the MK rate. 

Correspondence training was found to be a less costly method 
of training to the E4 level of proficiency. 

C. SUMMARY OF LITERATURE REVIEW 

Training effectiveness is a complex issue with numerous 
significant variables. The task of assessing the effect of 
the independent variables has been attempted by many in the 
past, with a variety of resultant findings. Researchers have 
had to use the measures of effectiveness available to them 
and attempt to account for the threats to validity inherent 
i n each mode 1 . 

The next section will develop a research methodology that 
takes account of the variables that are characteristic of 
each training program and which may account for some of the 
variability in the effectiveness of each program of 
instruction. The variables which account for training 
program effectiveness and are not inherent in the actual 
training method will also be assessed in relation to the 
threat to validity they pose to the model. 
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V. 



RESEARCH METHODOLOGY 



The comparative methodology for this study was based on 
the strategy of performing secondary archival research of the 
USCG Performance Management Information System CPMIS). As 
pointed out in the last two chapters, Job performance was 
dependent, in some part, on the Initial rate training 
received, which enables personnel to perform the tasks that 
make up the job. The relative effectiveness of the training 
methods used to incorporate basic rate skills, knowledge and 
abilities in the Coast Guard enlisted personnel can be 
measured using the criteria of performance evaluations and 
rate of advancement. 

This chapter will outline the research model used in the 
study, specify the data to be collected, assess threats to 
validity and the scope of the methodology. 

A. STATIC GROUP COMPARISON 

The model used in this study was based on a static group 
comparison and is shown in Table 3. Enlisted personnel 
currently in the database were sorted by rate and training 
method and then again by paygrade. To control for variance 
in job performance as a function of the variance in aptitude, 
a range of ASVAB composite scores was defined which contained 
an appropriate mix of "A" school and striker graduates. The 
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data from these records were used to make subsequent 
comparisons between the groups. 



ADVANCEMENT MODEL 
TABLE 3 



Striker Program 



Serv i ce 
Separat i ons 



E4 






V 

E5 


(Time 
to E5 


to Advance 
from E4) 


V 

E6 


(Time 
to E6 


to Advance 
from E4> 



"A" School Program 

4 



IOC 

1 . 



Service 
E5 — » Separat i ons 



E6' 



B . DATA 

With records selected for each training method and sorted 
by rate, paygrade and ASVAB subtest composite range, data was 
then extracted and used as criteria to evaluate differences 
in job performance between the groups, 

1. Enlisted Performance Evaluations 

The Enlisted Performance Evaluation (Appendix C) is 
conducted twice yearly for all Coast Guard personnel E4-E6. 
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It is a behavioral ly anchored rating scale from one to seven 
and measures six composites of human behavior. While all six 
components are significant with respect to the overall 
productivity of the individual, only the work factor can be 
linked with any confidence to the initial rate training 
program the individual encounters. The work factor is broken 
down into eleven rated categories including knowledge, 
determining priorities, using resources, monitoring work, 
guidance required, keeping supervisor informed, workmanship, 
meeting deadlines, learning from experience, observing safety 
practices and stamina. While some of these may not be 
dependent upon training method, it is the best overall 
measure currently available to assess job performance. The 
work factor score is determined by the member's supervisor 
and has a range of 11 (lowest performance) to 77 (highest 
performance). The form has been designed so that the 
majority of Coast Guard personnel will be described by the 
characteristics in blocks 3 to 5. Again, since the 
assignment process is random, there is no expected bias in 
the use of this instrument as a measure of job performance. 

The work factor was extracted from al 1 records for 
each paygrade and compared, e.g., work factors for all MK2 
(E5> personnel, work factors for all BM3 (E4) personnel. A 
separate variance T-test was used to assess if a significant 
difference in average job performance exists between the 
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groups as measured by the work factor of the enlisted 
evaluation system. 

2. Rate of Advancement 

The rate of advancement is defined as the length of 
time required for an Individual to attain a higher rate. The 
advancement process, previously detailed in section II. C., is 
characterized by a servicewide competition and a number of 
performance hurdles. 

Each record was polled to determine the length of 
time taken to reach each paygrade from the initial attainment 
of E 4; e.g., if the record of an E6 was in the BM "A" school 
population, then the length of time taken to attain E5 was 
used in determining the average time to advance to E5 within 
the BM "A" school graduate population. This average rate of 
advancement was then compared to the average rate of 
advancement for the BM striker population to determine if a 
significant difference exists. 

C. THREATS TO INTERNAL VALIDITY 
1 . Se 1 ect ion Bias 

Acceptance to "A" school is contingent upon a minimum 
ASVAB subtest composite score as detailed in Table 1. Any 
differences in job performance as a result of teaching 
methodology would be difficult to analyze due to selection 
bias present in the system. This effect is attenuated by 
selecting groups which are comparable in aptitude as measured 
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by the ASVAB minimum set for "A" school selection. It is 
assumed that there are strikers who met the minimum ASVAB 
requirements for "A" school, but decided to strike the rate 
at their unit rather than attend a formal school. The 
aptitude differential between groups could have an impact on 
servicewide exam performance which accounts for 44% of the 
SWE composite that is used in the advancement process. With 
aptitude controlled, the behavioral differences between the 
groups can be attributed in greater weight to the different 
teaching methodologies to which each group is exposed. 

2. Motivation 

There are several theories which attempt to describe 
the sources of human motivation ranging from the satisfaction 
of basic needs to expectancy/valence which attempts to 
account for both intrinsic and extrinsic factors. Varying 
levels of motivation will affect personnel behavior at all 
levels beginning with the learning process during training 
and continuing to job performance in the field. Data which 
relects the differences in motivation in personnel are not 
currently available. For the purposes of this study, it is 
assumed that since all personnel are assigned without bias 
following the attainment of E4 , the external motivation 
resulting from the individual's environment (i.e., 
supervisor's characteristics, work conditions) is distributed 
equally through both groups. There are no assumptions made 
regarding the distribution of intrinsic motivation within 
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each group. It could be that strikers, since they are 
involved in a self-paced curriculum, are more se 1 f-mot i vated 
which could be a significant variable affecting job 
performance and rate of advancement. This will be addressed 
again in the conclusion. 

3. Experimental Mortality 

This study does not include members who separated 
from the service during the period of the study. While it 
may not be a factor, one group or the other could be 
experiencing significant mandatory separation rates due to 
unsatisfactory performance. If those members are no longer 
part of the comparison groups, (assuming they were in the 
initial comparable ASVAB criteria range) the findings could 
be skewed from the loss of low end observations and will be 
less indicitive of the true population behavior. Since the 
starting point of the study is the attainment of E4 , no data 
have been obtained on the dropout rate from the striker 
program as compared to the "A" school programs. This is more 
a measure of the efficiency of the program than it is of the 
effectiveness, but would provide a more descriptive picture 
of the overall implementation of both training programs. 

4. Sample Contamination 

While on the "A" school waiting list, it is likely 
that some members are gaining rate experience at their parent 
unit. This gives them a base of practical experience in the 
field which enables them to Incorporate more of the training 
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they receive at "A" school. Thus, some "A" school graduates 
will be contaminated by on-the-job training previously 
received in that rate. There are no records to account for 
the experience level of the entry level "A" school trainee. 
It is rare for an "A" school dropout to successfully strike 
the rate. 

D. SCOPE OF METHODOLOGY 

This study is based on a methodology of behavioral 
comparison and is intended to be used as an indicator of the 
relative effectiveness of the two primary enlisted rate 
training methods. It does not attempt to analyze the 
implementation of the training programs at a micro level. 
Rather, there are general statements made regarding the 
characteristics of each training method. There is also no 
attempt to attach program costs to each training method, 
although a cost effectiveness study of training would 
certainly be a viable area for follow-on research. 



32 



DATA AND ANALYSIS 



VI . 

Data from CG PMIS was collected on current active duty 
personnel. Criteria previously described were applied to 
6,200 observations which resulted in 1,264 observations that 
were further sorted by rate, rank and training methodology. 
Data from the resultant observations were then compared to 
determine if differences existed for average work factor or 
for rate of advancement as a function of training 
methodol ogy . 

A. AVERAGE WORK FACTOR 

The average work factor represents the average mark 
obtained within the work factor composite of the Enlisted 
Performance Evaluation form for all marks given in three 
marking periods (Fall 1987, Spring 1988 and Fall 1988). The 
data analyzed represent one aspect of rated job performance 
of those individuals who met the criteria for the comparison. 
One problem encountered in the data analysis was the number 
of missing values associated with this factor from the 
database. In many cases, the 1,264 observations which 
qualified for the comparison, based on previously described 
criteria, did not have enlisted evaluation marks and were 
treated as missing values. It is assumed that the missing 
values are randomly distributed between the two populations 
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and does not have a significant impact on the results of the 
comparison. In other words, the data presented should be 
representational of the complete data set and not create a 
concern for validity of the comparisons to the "true" 
behavior of the populations. 

1. Results of Average Work Factor Comparison 
Table 4 details the results of T-test procedures 

applied to the data. The null hypothesis assumes that the 
average work factor is equal for both groups. The results of 
the T-tests indicate that the null hypothesis cannot be 
rejected. In other words, there is no significant difference 
in average work factor between the two groups across rates at 
either the E4 or E5 level. 

2. Discussion 

The results of equal average work factor between the 
groups were expected for the following reasons: 
a. Expected Range of Performance 

Performance standards detailed in the Enlisted 
Performance Evaluation form (Appendix C) are designed so that 
the majority of personnel being evaluated will fall between 3 
and 5. With this "expected" range thus established, 
evaluators may be reluctant to give marks outside the range 
unless a person's performance is truly exceptional in one 
direction or the other. 
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b. Common Minimum Performance Criteria 

Both groups are trained towards a set of common 
minimum performance criteria established in the EQCM. If 
both groups are actually attaining rate knowledge and skills 
according to the established criteria, they should be 
evaluated on-the-job as exhibiting the average behavior 
described for each mark, if positive transfer Is taking 
P 1 ace . 



The true comparison of effectiveness of training 
programs would be better established if data were obtained 
for personnel who left the service after attaining rate. 
Inadequate marks are one criterion for discharge from the 
Coast Guard for unsatisfactory performance. Of course, those 
people who have attrited may have contributed to an increased 
differential between the average marks for the two groups 
which might indicate a performance difference as a result of 
training method. 
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AVERAGE WORK FACTOR MARK 



TABLE 4 

RATE/ 
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A larger sample size would increase the power of 
these tests and also decrease the possibility that one 
outlier would skew the results. The SS E5 comparison of 
striker and "A" school methods only uses one striker mark and 
five "A" school marks and shows a significant difference at 
the .05 level of test significance. But with only one 
striker mark, there are too many explanations available to 
account for this differential that have nothing to do with 
training method. As a result, the comparisons with larger 
sample sizes are more credible and are more likely to be 
representative of the "true" behavorial differences between 
the groups as a result of training method. 

Previous studies have shown that aptitude does not 
necessarily predict actual job performance. Used in 
conjunction with the results of th i s comparison, it would 
seem likely that the criteria of minimum ASVAB composite 
required for "A" school used in this study could be relaxed 
and the T-test comparisons would again result in no 
significant difference in rated performance as a function of 
training methodology. The ASVAB selection criteria becomes 
more important in relation to the rate of advancement 
comparison in the next section. 

B. AVERAGE RATE OF ADVANCEMENT 

The average rate of advancement represents the average 
number of days taken to advance to E5 from the attainment of 
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E4. Of the strlkeable rates, only four are comparable due to 
population size constraints. 

1. Results of Average Rate of Advancement Comparison 
Table 5 details the results of the T-test procedures 

applied to the data. The null hypothesis assumes that the 
average rate of advancement is equal for both groups. The 
results of the T-test are: 

o There is a significant difference in the rate of 

advancement to E5 in both the BM and MK rates at the .05 
level of test significance (strikers advance almost 
twice as fast as "A" school graduates). 

o There is no significant difference in rate of 

advancement to E5 in either the SK or YN rates at the 
.05 level of test significance. 

o The distribution of advancement rate for the BM "A" 
school population is bimodal whereas the BM striker 
population exhibits a normal distribution. The 
remaining rate distributions are difficult to 
characterize due to low number of observations. 

2. Discussion 



The differential rate of advancement could result 

from the effects of a variety of influences. A review of 

advancement criteria detailed in section II. C. shows that 

rate of advancement is dependent upon: 

o Minimum performance qualifications (to qualify to 
participate in SWE competition). 

o Composite SWE score. 

o Time in service. 

o Time in rate. 

o Medals and awards. 

o Needs of the service. 
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